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Introduction 

State-level initiatives, along with accrediting agency requirements, are cited as important 
direct influences on institutions’ student assessment efforts (Ewell, 1993). More than 40 states 
have mandated assessment policies in place for public institutions of higher education. In 
response to such external demands, many colleges and universities have initiated some form of 
student assessment activity (Banta, Lund, Black, & Oblander, 1996; El-Khawas, 1995). There 
has been little research, however, to inform us if these activities have led to improved 
educational quality, despite the reality that most states cite improvements in teaching and 
learning as two primary goals of their assessment mandates. Furthermore, we do not know if the 
perceptions state policy makers hold toward the outcomes of their assessment policies match the 
activities done, and the perceptions held, at the institutional level. 

Based on some preliminary research conducted on behalf of the National Center for 
Postsecondary Improvement, we have found that there are some states in which a policy maker 
reports that their student assessment policy was intended to improve teaching and learning in 
their state, but, in fact, has not. The purpose of this paper is to examine states with mandated 
assessment plans in which policy makers report such discrepancies between their policy 
objectives and their policy outcomes on improving teaching and learning. We will compare 
these policy makers’ perceptions to the perceptions and activities of public institutional 
representatives in their state. Our three research goals are to gain a better understanding of 1) 
perceptions of campus representatives on how engaging in student assessment has affected the 
teaching and learning process on their campus; 2) perceptions of state higher education executive 
officers (SHEEOs) on how engaging in student assessment has affected the teaching and learning 
process at public institutions in their state; and 3) the extent to which these perceptions differ and 
why differences may exist. 
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Literature Review 

We have created a framework for analyzing student assessment based on a recent 
literature review (Peterson, Einarson, Trice, & Nichols, 1997). The framework includes five 
environments: 1) the external influences on student assessment; 2) institutional approaches to 
student assessment; 3) the role of institutional characteristics in determining institutional 
approaches to assessment; 4) the institutional support, assessment policies and practices, and 
culture and climate for assessment; and 5) institutional utilization of and impact from student 
assessment results. In this report, we are focusing on discrete parts within each of these 
environments — examining how state policy mandates relate to approaches to, practices 
regarding, institutional uses of, and impacts of student assessment in public institutions. 

Influence of the State on Assessment Activities 

State-level actions are believed to be an important influence on institutional engagement 
in student assessment efforts. The majority of state-level assessment efforts originated in the mid 
1980s following NIE’s Involvement in Learning and other national reports that called for 
assessment of student learning (Aper, Cuver, & Hinkle, 1990; Ewell, 1993). Since then, the 
number of states enacting student assessment initiatives has increased steadily. According to 
recent research conducted by the National Center for Postsecondary Improvement, all but four of 
fifty responding states reported some type of student assessment activity (Cole, Nettles, & Sharp, 
1997). 

Several studies have found that state requirements have spurred institutional activity. 
Hexter and Lippincott’s (1990) review of three surveys of institutional engagement in student 
assessment activities found that external mandates from accrediting agencies, state legislatures, 
and boards of regents played a prominent role in prompting assessment initiatives, particularly at 
public institutions. In addition, half of the institutions reported utilizing student assessment 
results for reports to state agencies. Ewell (1993) stressed that state requirements have spurred 
activity both in assessment of entering students and in testing students’ knowledge within the 
major. 

However, other studies have found that institutions do not always comply with state 
requirements. Ewell & Boyer (1988) used interviews with state and institutional representatives 
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from five states engaged in state-mandated assessment to explore the relationship between state 
approaches to assessment and institutional responses. Unclear communication and lack of trust 
were noted as significant problems. State officials and institutional representatives were likely to 
interpret assessment terminology differently and seemed uninformed of differences in political 
and academic timetables. Three general categories of institutional responses were observed: 
some institutions actively resisted assessment mandates provoking state measures to obtain 
compliance; most institutions provided minimal information, giving the state only what it was 
presumed they wanted; and a few institutions took a proactive stance, developing and then 
selling locally developed assessment programs to state officials. In Ewell and Boyer’s view, 
differences in institutional response were not related to institution type but to academic 
leadership choices. Institutional leaders often have a more difficult time selling assessment to 
internal constituents when it is perceived as an external mandate from either the state or an 
accreditation association (Ewell, 1993; Muffo, 1992). 

Assessment Impacts on Teaching and Learning 

Although engaging in student assessment activities is thought to contribute to several 
institutional impacts, in this study we are focusing on how such engagement impacts the teaching 
and learning process. The literature offers little direct evidence of changes in students’ academic 
performance as a result of student assessment efforts (Banta & Associates, 1993; Banta, Lund, 
Black, & Oblander, 1996). A few institutions have reported improvements in student 
performance on standardized examinations (Bowyer, 1996; Krueger & Heisserer, 1987; 

Magruder & Young, 1996) and increases in students’ mean grade point averages (RiCharde et al., 
1993). The following impacts on currently enrolled students have been reported by some 
institutions: increased student involvement as reflected in time spent studying, frequency of 
academic-oriented discussion with other students, library usage (Krueger & Heisserer, 1987), 
interaction with faculty (Williford & Moden, 1993) and participation in class discussions 
(Friedlander, Murrell, & MacDougall, 1993); enhancement of students’ attitudes toward their 
education including greater satisfaction with their instructional experiences and institutional 
services (Krueger & Heisserer, 1987, Williford & Moden, 1993), finding classes more 
challenging (Katz, 1993), and reporting greater seriousness about their studies (Ciereszko, 1987); 
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increases in freshman retention (Blanzy & Sucher, 1992; Walleri & Seybert, 1993); and an 
increased proportion of graduates pursuing further education (Young & Knight, 1993). 

The impact of assessment on student performance has received less consideration in 
multi-institutional research. Compared to other uses of assessment information mentioned in the 
1989 Campus Trends survey (El-Khawas, 1989), providing feedback to students was the use 
reported least often. Conversely, Johnson and colleagues (Johnson, Prus, Andersen, & El- 
Khawas, 1991) found improved student effort and feedback were among the main benefits that 
institutions with comprehensive student assessment programs associated with having 
implemented assessment efforts. Interviews with directors of fifteen pilot projects on student 
assessment undertaken in the California State University system revealed uncertainty regarding 
whether gains in student achievement had been realized as a result of assessment efforts 
(CSUITL, 1993). No direct evidence was found that related assessment efforts or the provision 
of assessment information with changes in student performance. 

Descriptions of assessment practices at various institutions contend student assessment 
efforts have stimulated changes in the teaching methods and course-embedded assessment 
methods used by faculty (Banta, Lund, Black, & Oblander, 1996; Banta & Moffett, 1987; Lang, 
1993). Associated changes in the nature of course-embedded assessments of student learning 
include the use of portfolio assessment (Katz, 1993; Loacker & Mentkowski, 1993; Young & 
Knight, 1993) with increased emphasis on critical thinking, and oral and written communication 
skills (Katz, 1993; Williford & Moden, 1993; Young & Knight, 1993). Institutions report having 
made the following changes in teaching methods and learning activities based on student 
assessment information: increased use of collaborative and cooperative teaching methods 
(Friedlander, Murrell, & MacDougall, 1993); introduction of computer applications (Walleri & 
Seybert, 1993; Young & Knight, 1993); development of an undergraduate research program 
(Young & Knight, 1993); incorporation of out-of-class learning activities in course requirements 
(Friedlander, Murrell, & MacDougall, 1993); and expanded internship and clinical experience 
opportunities (Young & Knight, 1993). 

Evidence from multi-institutional studies regarding the impact of student assessment on 
faculty instructional and assessment practices is less available. Relative to other possible 
institutional uses, two-year institutions in Cowart’s (1990) study infrequently reported using 




7 



6 



student assessment information for the development of alternative instructional modes. 

Similarly, directors of fifteen pilot projects on student assessment in the California State 
University system were less certain about improvements in faculty teaching achieved as a result 
of assessment projects than they were about other project outcomes (CSUITL, 1993). We did 
not locate any comparative research regarding institutional or faculty engagement in student 
assessment efforts and changes in faculty approaches to instruction or assessment. 

Methods 

For this study, we chose three states as contexts for pursuing our research goals. The 
information we used to choose these three states stems from several sources. During 1996-97, 
we reviewed the literature of prior research on state assessment and regional accreditation 
policies, examined policy documents from each of the 50 states and the District of Columbia, 
discussed assessment policies and procedures with state higher education executive officers, and 
requested (and received) reactions from state higher education executive officers and accrediting 
officials to draft reports about their state/association. In December 1997 we mailed the State 
Higher Education Assessment Questionnaire to all 50 state higher education executive officers — 
to date we have received 38 responses. 

In January 1998, we sent the Inventory of Institutional Support for Student Assessment to 
each Chief Academic Officer at 2,528 U. S. postsecondary institutions. We received 1,393 
responses, for a response rate of 55%. Of these 1,393 institutions, 885 are public, which equates 
to a 60% response rate from all the public institutions we surveyed. Most surveys were 
completed by the Chief Academic Officer at each institution. This survey is a comprehensive 
inventory of external influences on undergraduate student assessment; institutional approaches to 
student assessment; patterns for organizational and administrative support for student 
assessment; assessment management policies and practices; and the uses and impacts of 
assessment information. We created a database to house the results of this survey— also included 
in this database are data from both the Integrated Postsecondary Education Data System and the 
State Assessment and Regional Accreditation Policies study. 

Our main criteria in choosing states was that the state higher education executive officer 
had reported that the improvement of teaching and/or learning was a policy objective of their 




8 



7 



statewide assessment plan, and had also reported that the improvement of teaching and/or 
learning was not an achieved outcome of their assessment requirements. Once this overriding 
criterion was met, we chose states in which their SHEEO both had provided fairly detailed 
responses to our state survey and had provided extensive documentation during our document 
collection phase. Our final criteria focused on selecting three states from three different 
accrediting regions and with at least two different governance structures. As we only included 
public institutions in our analyses, the n’s were: State A = 14, State B = 10, and State C = 26. 
Table 1 lists the types of institutions represented in our analysis. 



Table 1 








Public Institutions in States A, B, & C 








State A 


State B 


State C 


Accrediting Region 


North Central 


New England 


Southern 


Authority Structure 


Coordinating Regulatory 


Coordinating Regulatory 


Consolidated Governing 


Institutional Type 








Associate of arts 


8 


6 


19 


Baccalaureate 


2 


1 


0 


Master’s 


1 


3 


1 


Doctoral 


2 


0 


3 


Research 


1 


0 


3 


Total 


14 (of 27) 


10 (of 18) 


26 (of 37) 



Once we had chosen the three states, we performed both qualitative and quantitative 
analyses on our data. Qualitatively, we content-analyzed the student assessment documentation 
for each state and the responses that each SHEEO provided to the open-ended questions on our 
state survey. These analyses gave us a full picture of the policy objectives, the policy outcomes, 
and the reasons the SHEEOs gave for the mismatch between the objectives and the outcomes for 
improving teaching and learning. 

Quantitatively, we examined frequency distributions and cross-tabs for the public 
institutions in each of our three states. Due to the low N’s within each of our three states, we 
could not use ANOVAs to compare the results for institutions in our three states to those for all 
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public institutions. However, we do display these comparisons as a benchmark for how 
institutions in our three states compare to the rest of the country. 

Throughout this report, we will refer to “campus representatives” and to “SHEEOs.” The 
campus representatives are the respondents to our institutional survey: “Inventory of Institutional 
Support for Student Assessment.” While this instrument was addressed to the Chief Academic 
Officer on each campus, it was completed by the person with the most knowledge and experience 
regarding student assessment. The SHEEOs are the state higher education executive officers 
who responded to the State Higher Education Assessment Questionnaire. 



Results 

For each state, we present six sections. In the first section, we describe the policy 
background for that state, focusing on the perception of the SHEEO on the match between the 
objective and the outcome of improving teaching and learning through student assessment. In 
the second section, we discuss how the public institutions in the state perceive the state policy. 

In the third section, we present the approaches to student assessment used by public institutions 
in the state. The fourth section consists of a discussion on the amount of support evidenced by 
public institutions in the state for conducting student assessment. The fifth section discusses the 
impacts of conducting student assessment, as reported by the institutional representatives. In the 
summary section, we will compare the SHEEOs’ perceptions to those of the institutional 
representatives. 

State A 

Policy Background 

State A had a student assessment system in place based on statute from 1989 through 
1996. The statute was specifically designed to hold institutions accountable, both for the 
efficient use of state funds as well as for the provision of quality education. This statute did 
include a minor appropriations component; the state’s higher education commission was 
authorized to withhold up to two percent of an institution’s annual state appropriation if the 
institution did not comply with the terms of the statute. Under this statute, institutions were 
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required to measure the increase in knowledge and skills between entrance and graduation, with a 
focus on general education and learning in the discipline. Institutions also needed to measure 
student satisfaction, after-graduation performance, retention rates, and graduation rates. 

In 1996, State A’s legislature dramatically expanded the appropriations component with a 
new statute—which replaced and superseded the old statute and its requirements--that mandated 
the creation of a performance funding, or quality indicator, system. Among the areas to be 
evaluated by the quality indicator system are institutional performance, student satisfaction and 
success, employer satisfaction, and systemwide performance. One of the indicators requires the 
presence of a functioning student assessment system. Under this new statute, institutions have 
been required to report both their results based on their assessment system, and how they plan to 
improve these results. 

State A’s assessment statute has four objectives: (1) to increase accountability to the 
public; (2) to increase fiscal responsibility; (3) to improve teaching; and (4) to improve learning. 
By contrast, the SHEEO in State A reports three outcomes: (1) the improvement of student 
learning; (2) the promotion of planning on campuses; (3) the improvement of academic program 
efficiency. According to the academic officer, the improvement of teaching has not been an 
outcome of State A’s assessment statute. One reason suggested by the SHEEO for this failure is 
the difficulty to measure and/or demonstrate an improvement in teaching to the state legislature. 
In other words, it may be a problem of perception (or definition) rather than of an unmet 
objective. Institutions may, in fact, be seeing incremental improvements in teaching, but they 
may be too small, too subtle, or too esoteric for legislatures to comprehend. Data from the 
institutions themselves should help to explore further this potential gap between policy objective 
and outcome in State A. 



Institutional Perceptions of Policy 

Before examining the assessment activities and impacts for the institutions in State A, it 
should be instructive to look at the institutional perceptions of the state policy. Our analysis 
focused on the 14 public institutions in State A who responded to ISSA survey. Details of the 
perceptions of these 14 institutions are presented in Table 2. While most of the institutions agree 
that there is a state plan for assessment (12) and all institutions agree that responding to state 
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reporting requirements is an important reason to engage in assessment, there is still confusion 
regarding the details of the state policy. Of the 12 institutions who agree that there is a state 
policy, six reported that the assessment plan was developed solely by state officials and six 
reported that it was developed jointly between state officials and campus representatives. (There 
was no clear pattern of institutional types represented in these differing views.) Most agreed that 
the state requires evidence that an assessment plan is in place (11), measurement of state- 
mandated indicators (10), and use of institutionally-devised indicators (8). However, only half of 
the institutions agreed that the state required evidence of institutional use of assessment data. 



Table 2 

Institutional Perceptions of State Policy in State A 

Number 

Responding 

Perception N=14 



Meeting state reporting requirements is important reason to conduct assessment 14 

There is a state plan requiring institutions to conduct assessment 12 

• Assessment plan was developed solely by state officials 6 

• Assessment plan was developed jointly between state and campus reps 6 

State plan was reason to initiative assessment 7 

State plan increased involvement with assessment 7 

State plan had negative influence on assessment 2 

State requires evidence that an assessment plan is in place 1 1 

State requires measurement of state-mandated indicators 10 

State requires use of institutionally-devised indicators 8 

State requires evidence of institutional use of assessment data 6 



Institutional Approaches to Student Assessment 

The approaches that respondents use to gather student assessment data are listed in Table 
3. Based on the mean scores on three factors — cognitive, affective, and post-college assessment- 
-institutions in this state are collecting data for at least some students on post-college outcomes, 
but not all institutions are collecting data on either affective or cognitive competencies of their 
current students, despite the state mandate for assessment. The factor “affective competencies” 
includes student satisfaction, which was mandated by the state— all of the institutions in State A 
do report collecting data on student satisfaction. 
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Table 3 

Approaches to Student Assessment in State A 



Approach 


Mean for Public Institutions 
in State A 


Mean for all 
Public Institutions 


Cognitive Assessment 


1.64 


1.62 


Affective Assessment 


1.75 


1.74 


Post-college Assessment 


2.36* 


2.27 


Number of Instruments Used 


8 


9 


Student-Centered Methods 


1.28 


1.30 


Externally-Oriented Methods 


2.34** 


2.06 


Number of Studies Conducted 


2 


2 



*The scale for these three factors is 1 = not collected, 2 = collected for some students, 3 = collected for many 
students, and 4 - collected for all students. 

**The scale for these two factors is l=not used, 2=used in some units, 3=used in most units, 4=used in all units. 



In addition to traditional instruments, the ISSA instrument asked institutions if they used 
student-centered methods for assessment, including portfolios, performance in capstone courses, 
and observations. Institutions in State A scored a 1 .28 on this student-centered method factor. 
Apparently, these institutions are not making great use of student-centered methods for 
assessment. These institutions are more likely to be using externally-oriented than student- 
centered methods for assessment. The policy in this state explicitly requires institutions to 
measure employer satisfaction., and 1 1 of the 14 institutions report collecting data on students 
from employers. Institutions in State A appear to be responsive to the state policy, in terms of 
measuring both student and employer satisfaction. In addition, institutions in this state do not 
look remarkably different from public institutions throughout the country. However, the level of 
assessment conducted in this state is somewhat low — based on the cognitive, affective, and post- 
college factor scores— which is surprising given the state mandate. 
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Institutional Support for Student Assessment 

Table 4 presents frequency information on institutional support for student assessment in 
State A, along with percentages for all public institutions. Although these percentages are not 
compared for statistically significant differences, they provide an idea of how institutions in State 
A behave within the context of all public institutions. While most institutions in State A reported 
that improving faculty instructional performance is an important reason to conduct assessment, 
two institutions did not believe this purpose is important, indicative of a lack of a uniform 
purpose to improve teaching via student assessment throughout the state. Furthermore, only one- 
quarter of the respondents replied that they have a faculty governance committee that addresses 
assessment, indicating a lack of a formal process across institutions for faculty to discuss student 
assessment issues. For all public institutions, 50% responded that they had a faculty committee 
that addressed assessment. Respondents in State A also reported that, on average, 50% of their 
existing faculty governance committees were supportive of student assessment. For all public 
institutions, 65% of their governance committees are reported to be supportive of student 
assessment. Not only may there be less support in State A, but if only 50% of the faculty 
governance committees are supportive of student assessment, it is likely that on only half of 
these campuses are faculty taking student assessment seriously or considering its impact for their 
teaching. Nonetheless, most campuses in State A are providing opportunities for professional 
development for faculty in the form of conference funds and workshops. More than half of all 
institutions require at least some of their faculty to engage in student assessment. 
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Table 4 

Support for Student Assessment in State A 


Percentage of 
Institutions in State A 


Percentage of 
all Public Institutions 


Support For Assessment 


N=14 


N=885 


Refer to assessment in mission 


7% 


23% 


Engage in assessment to improve faculty performance 


85% 


93% 


Formally adopted policy requiring assessment in all units 


57% 


51% 


Faculty governance committee addresses student assessment 


25% 


50% 


Faculty governance committees are supportive of assessment 


50% 


65% 


Conference funds are available for faculty 


100% 


87% 


Workshops are offered for faculty 


71% 


79% 


Faculty required to receive assessment training 


57% 


59% 


Faculty assistance is offered (i.e. paid leaves, stipends, course 
reductions) 


50% 


53% 



The survey also asked respondents to indicate how student assessment relates to faculty 
evaluation and rewards. Table 5 displays the institutional responses from State A. While all 
institutions in State A encourage faculty to assess their students, relatively few tie such 
assessment to tenure, promotion, or salary increases. In comparison with all public institutions, 
fewer institutions in State A are considering student performance in faculty evaluations or 
assessment participation in promotion. Furthermore, fewer institutions in State A are publicly 
awarding faculty for their participation in assessment. 
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Table 5 

Faculty Evaluation and Rewards in State A 



14 





Percentage of 


Percentage of 




Institutions in 


all Public 


Faculty Evaluation and Rewards Policies 


State A 


Institutions 


Faculty encouraged to assess students 


100% 


93% 


Scholarship on student assessment considered for tenure 


38% 


43% 


Evaluation for salary increases considers student performance 


21% 


22% 


Willingness to participate in assessment considered in promotion 


21% 


39% 


Faculty receive awards for assessment 


21% 


33% 


Hiring process considers assessment experience 


15% 


38% 


Evaluation for promotion considers student performance 


14% 


30% 



Institutional Impacts of Student Assessment 

In addition to exploring institutional approaches to and support for student assessment, 
our survey allows us to examine the impacts institutions ascribe to their assessment efforts. This 
section of the survey was presented in two segments: institutional actions and institutional 
impacts. In regards to actions, institutions were asked whether they used student assessment data 
in various institutional activities. One of these activities was “modifying instructional or 
teaching methods.” In State A, nine of the 14 respondents (64%) responded that action had been 
taken to modify instructional methods and that student assessment data had been influential in 
this modification. This percentage compares to 59% for all public institutions. 

Survey respondents also reported whether they have monitored how student assessment 
impacts on various faculty activities, such as campus discussions of undergraduate education, 
interest in teaching, and the changes in teaching methods. Table 6 displays how respondents in 
State A answered these questions. 
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Table 6 

Impacts of Student Assessment on Faculty in State A 

Number of 
Number of Institutions 

Institutions Finding Positive 

Impact Monitoring Impact 



Changes in Instructional Methods Used 



8 



7 



Discussions of Undergraduate Education 



8 



6 



Faculty Interest in Teaching 



3 



1 



Faculty Satisfaction 



4 



0 



Apparently, several institutional representatives in State A have found that student assessment 
has led to changes in instructional methods used by faculty as well as to an increased focus on 
undergraduate education. Fewer institutions are examining whether student assessment has 
impacted faculty interest in teaching and faculty satisfaction; those that have monitored these two 
realms have not, on the whole, found positive impacts. 

Summary for State A 

In terms of policy perception, there is confusion in State A regarding how the state plan 
for assessment was developed and what it requires. While the types and frequency of student 
assessment data collected mirrors all public institutions throughout the country, institutions in 
this state are collecting neither cognitive nor affective data on the majority of their students. 
Neither are they making extensive use of student-centered methods for data collection, such as 
portfolios, observations, and capstone courses. 

In terms of institutional support for student assessment, most respondents stressed that 
they engage in student assessment to improve faculty instructional performance, indicating that 
they share this policy objective. Institutions in State A are providing opportunities for 
professional development for faculty to learn about student assessment. However, they are not 
tying assessment participation or results to faculty promotion or tenure considerations. Finally, 
less than half of faculty governance committees in State A are supportive of student assessment. 

Respondents from State A do believe that assessment has impacted teaching by focusing 
faculty attention on undergraduate education and on changing teaching methods. The 
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implication is that student assessment has had positive impacts on teaching — an opposite belief 
to that held by SHEEOs in the state. While respondents to the survey believe that assessment has 
improved teaching, their answers also provide a glimpse of their assessment activity level. The 
depth of assessment activity and the faculty support for assessment are all somewhat 
questionable in State A. Perhaps legislators are aware of the relative lack of support, which may 
be adding to their speculations of the impact assessment has on teaching. 

State B 

Policy Background 

State B has required institutions to assess institutional effectiveness and make a report of 
their progress since 1988. Institutions were to report to the state board of higher education. In 
developing this policy, the state board noted that the overall objective of assessment was to 
enhance the quality of instruction and student performance. In 1993, State B’s legislature 
mandated biennial assessment of institutional effectiveness, and further required that the results 
of this assessment be reported not only to higher education officials but also to appropriate 
committees in the state legislature. Three years later, in 1996, the state board of higher education 
issued guidelines for these reports. They stipulated that institutions assess general education and 
basic skills, as well as student performance, attainment, and development. The 1993 statute and 
1996 reporting guidelines seem to be the state’s attempt to provide legislative force behind the 
1988 policy. 

State B’s assessment statute has four objectives: (1) to improve teaching; (2) to improve 
student learning; (3) to promote planning on campuses; and (4) to improve academic program 
efficiency. By contrast, the SHEEO in State B reports two assessment policy outcomes: (1) the 
promotion of planning on campuses; and (2) the improvement of academic program efficiency. 
According to him, neither the improvement of teaching nor the improvement of student learning 
has been an outcome of State B’s assessment statute. Among the reasons offered by the state 
officer for this disconnect are a lack of resources necessary to implement assessment activities 
and a lack of commitment to assessment on the part of faculty. According to the state officer, the 
relationship between the assessment statute and the improvement of teaching and learning is 
weak because the primary emphasis is on feedback from graduate surveys, as well as program 
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reviews. It should be informative to compare the findings of institutional-level data to the state 
officer’s perceptions for State B. 

Institutional Perceptions of Policy 

Table 7 presents information on the institutions’ perceptions of the state policy. All of 
the 10 public institutions in State B who responded to the ISSA survey agreed that there was a 
state plan for assessment and all responded that meeting state reporting requirements was an 
important reason for conducting student assessment. While these responses indicate a fairly 
uniform level of knowledge throughout the state, 6 institutions reported that the plan had been 
developed jointly between campus and state officials and 4 institutions reported that it had been 
developed only by state level officials. There was no clear pattern of institutional types 
represented in these differing views. 



Table 7 

Institutional Perceptions of State Policy 

Number 

Responding 

Perception N=10 



Meeting state reporting requirements is important reason to conduct assessment 10 

There is a state plan requiring institutions to conduct assessment 10 

• Assessment plan was developed solely by state officials 6 

• Assessment plan was developed jointly between state and campus reps 4 

State plan was reason to initiative assessment 5 

State plan increased involvement with assessment 8 

State plan had negative influence on assessment 1 

State requires evidence that an assessment plan is in place 9 

State requires measurement of state-mandated indicators 4 

State requires use of institutionally-devised indicators 6 

State requires evidence of institutional use of assessment data 6 



Almost all of the institutional representatives (n=9) in State B agreed that the state 
requires evidence that an assessment plan is in place. Consensus on other aspects of the state 
plan, however, are mixed. Of the 10, 4 institutions believe that state reporting requirements 
include evidence of measurement of state-mandated indicators, 6 institutions believe that the 
state reporting requirements include use of institutionally-devised indicators, and 6 institutions 
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believe that the state reporting requirements include evidence of institutional use. Evidently, 
there is some confusion regarding what the state plan requires. 

Institutional Approaches to Student Assessment 

Respondents to the ISSA survey categorized both the types of assessment data they were 
collecting and the extent to which they collected this data. This data is presented in Table 8. 
Institutions in this state are collecting data for at least some students on post-college outcomes, 
but not all institutions are collecting data on either affective or cognitive competencies of their 
current students, despite the state mandate for assessment. It is likely that SHEEOs are aware 
that more effort is apparently spent on collecting data on former students and, although this 
pattern is not unique to this state, it may not be what policy makers had intended by their student 
assessment policy. 



Table 8 

Approaches to Student Assessment in State A 



Approach 


Mean for Public Institutions 
in State B 


Mean for all 
Public Institutions 


Cognitive Assessment 


1.53 


1.62 


Affective Assessment 


1.85 


1.74 


Post-college Assessment 


2.28* 


2.27 


Number of Instruments Used 


9 


9 


Student-Centered Methods 


1.23 


1.30 


Externally-Oriented Methods 


2.15** 


2.06 


Number of Studies Conducted 


2 


2 



♦The scale for these three factors is 1 = not collected, 2 = collected for some students, 3 = collected for many 
students, and 4 = collected for all students. 

**The scale for these two factors is l=not used, 2=used in some units, 3=used in most units, 4=used in all units. 



On average, institutions in State B used 9 instruments (both institutionally and externally 
developed) to collect student assessment data, which is the same number used on average by 
public institutions throughout the country. In addition to traditional instruments, the ISSA 
instrument asked institutions if they used student-centered methods for assessment, including 
portfolios, performance in capstone courses, and observations. The ISSA instrument also asked 
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about the use of externally-oriented assessment methods, such as interviews with employers and 
alumni. Institutions are more likely to be using external than student-centered methods for 
assessment (see Table 8), as perceived by the SHEEO. 

Institutional Support for Student Assessment 

Table 9 presents data regarding institutional support for student assessment in State B. 

All responding institutions believe that improving student performance is an important purpose 
for conducting student assessment, and almost all believe that improving faculty performance is 
an important purpose. Apparently, institutions in State B share the goals of the SHEEO to 
improve both teaching and learning through student assessment. Respondents in State B 
reported that, on average, 67% of their faculty governance committees were supportive of student 
assessment and 44% of the institutions responded that their students were supportive of 
assessment. While these percentages are slightly higher than are those for all public institutions, 
support for assessment is not pervasive in this state. Furthermore, only half of the campuses 
reported that there was a formal faculty committee that addressed student assessment issues. 
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Table 9 

Support for Student Assessment in State B 


Percentage of 
Institutions in State B 


Percentage of 
all Public Institutions 


Support For Assessment 


N=10 


N=885 


Refer to assessment in mission 


20% 


23% 


Engage in assessment to improve faculty performance 


89% 


93% 


Engage in assessment to improve student achievement 


100% 


98% 


0 

Formally adopted policy requiring assessment in all units 

6 


50% 


51% 


Faculty governance committee addresses student assessment 


50% 


50% 


Faculty governance committees are supportive of assessment 


67% 


65% 


Students are supportive of assessment 


44% 


33% 


Provide a line item operating budget allocation to support 
assessment 


20% 


50% 


Conference funds are available for faculty 


86% 


87% 


Workshops are offered for faculty 


87% 


79% 


Faculty required to receive assessment training 


60% 


59% 


Faculty assistance is offered (i.e. paid leaves, stipends, course 
reductions) 


50% 


53% 



Table 9 also presents information on professional development efforts in State B. Most 
institutions provide conference funds and workshops, and many require faculty to receive 
training in student assessment. In half of the responding institutions, faculty assistance is 
available (i.e. paid leaves, stipends, course reductions) for those who would like to learn more 
about assessment. These percentages are extremely similar to those for all public institutions. 
These percentages indicate that there are resources available at most institutions in State B to 
support faculty in their assessment efforts. However, only two of the ten public institutions in 
State B have established an explicit line item operating budget allocation to support assessment. 
This number compares to 50% of all public institutions who have established an explicit line 
item operating budget allocation to support assessment. 
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The survey also asked respondents to indicate how student assessment relates to faculty 
evaluation and rewards. Table 10 displays their responses. While all institutions in State B 
encourage faculty to assess their students, relatively few tie such assessment to tenure, 
promotion, or salary increases. In comparison with all public institutions, fewer institutions in 
State B are considering student performance in faculty evaluations or assessment participation or 
scholarship in promotion. No institutions in State B are rewarding faculty for assessment. 
However, many more institutions in State B are considering assessment experience in the hiring 
process. Perhaps this consideration indicates that institutions in State B realize that they need to 
increase their student assessment activities. 



Table 10 

Faculty Evaluation and Rewards in State B 



Faculty Evaluation and Rewards Policies 

Faculty encouraged to assess students 
Hiring process considers assessment experience 
Scholarship on student assessment considered for tenure 
Evaluation for salary increases considers student performance 
Willingness to participate in assessment considered in promotion 
Evaluation for promotion considers student performance 
Faculty receive awards for assessment 



Percentage of 
Institutions in 
State B 

100% 

63 % 

22 % 

11 % 

11 % 

11 % 

0 



Percentage of 
all Public 
Institutions 

93 % 

38 % 

43 % 

22 % 

39 % 

30 % 

33 % 



Respondents also reported on their student-oriented policies on student assessment (see 
Table 11). Most institutions actively involve students in the assessment process — all responding 
institutions provide students with individual feedback, 89% of the institutions inform the 
students about the purposes of assessment, and 67% of responding institutions require at least 
some of their students to participate in student assessment activities. Only 1 1% of responding 
institutions encourage student participation with incentives. In comparison with all public 
institutions, 88% of all responding public institutions require at least some of their students to 
participate in student assessment activities and 40% encourage students to participate with 
incentives. 
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Table 11 

Student-Oriented Policies on Student Assessment 


Percentage of 


Percentage of 




Institutions in 


all Public 


Student Policies 


State B 


Institutions 


Provide students with individual feedback 


100% 


85% 


Inform students about the purpose for assessment 


89% 


88% 


Require students to participate in assessment 


67% 


88% 


Encourage participation with incentives 


11% 


40% 



Institutional Impacts of Student Assessment 

In addition to exploring institutional approaches to and support for student assessment, 
our survey results allow us to examine the actions and impacts institutions ascribe to their 
assessment efforts. Responses to questions in the survey relating to improving the learning 
process are presented in Table 12. A majority of institutions in State B are using assessment data 
to modify instructional methods, academic programs, and support services. These initiatives 
demonstrate an effort to use assessment results to improve the learning process. However, as the 
SHEEO in State B stressed, program review and other related activities do not necessarily equate 
with improvements in the learning process. 

Table 12 

Institutional Actions Using Student Assessment Data 

Action Taken/Data on Action Taken/Data on 

Assessment Influential Assessment Influential 

Institutional Actions in State B for all Public Institutions 

Modify instructional methods 70% 60% 

Design/reorganize academic programs 70% 62% 

Modify academic support services 60% 63% 

Revise academic mission 20% 41% 

Revise gen ed curriculum 20% 57% 
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Survey respondents also reported whether they have monitored how student assessment 
impacts on various faculty activities. Table 13 displays how respondents in State B answered 
these questions. Of the institutions monitoring how assessment has impacted on instructional 
methods, all found a positive impact. Only a small number of institutions are examining whether 
assessment is impacting faculty in terms of their interest in teaching and their satisfaction in 
general. 



Table 13 

Student Assessment Impacts on Faculty Activities in State B 

Number of 
Number of Institutions 

Institutions Finding Positive 

Impact Monitoring Impact 



Changes in Instructional Methods Used 



4 



4 



Discussions of Undergraduate Education 



3 



1 



Faculty Interest in Teaching 



1 



Faculty Satisfaction 



0 



Survey respondents also reported how engaging in student assessment has impacted their 
students. Table 14 summarizes these responses. Less than a third of all public institutions in 
State B are monitoring the impact assessment has on student achievement in these four realms. 
Of those that are monitoring, not all are finding that their assessment initiatives have positive 
impacts on student achievement. Perhaps it is most troubling that only one of ten institutions are 
attempting to link their assessment activities to student grades. 



Table 14 

Student Assessment Impacts on Student Performance 

Number of 
Number of Institutions 

Institutions Finding Positive 

Impact Monitoring Impact 



Student retention or graduation rates 3 1 

Student satisfaction 3 2 

Student achievement on external exams 2 1 

Student grade performance 1 0 
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State B Summary 

According to the SHEEO in State B, the improvement of teaching and learning are 
objectives of that state’s assessment plan, but they are not outcomes. Two reasons suggested by 
the academic officer for this disparity are lack of campus resources to implement assessment and 
lack of faculty commitment to assessment. At the institutional level, only 1 1% of respondents in- 
state B indicated that promotions and/or salary increases are linked to student assessment or 
performance. Perhaps the state-level perception that there is little faculty commitment to 
assessment is a reflection of the very limited use of assessment as a part of any faculty rewards or 
incentives structure at the institutions. Further, only 20% of institutions have established an 
explicit line item in their budgets for assessment, which is much lower than the national average 
of 50%. This could help to explain the perception held by the state academic officer that there is 
a lack of campus resources used to implement assessment. Despite an apparent lack of resources 
dedicated to student assessment, many more institutions in State B are considering assessment 
experience in the hiring process for faculty positions. Perhaps this consideration indicates that 
administrators and faculty in State B realize that they need to increase their student assessment 
activities. 

State C 

Policy Background 

State C was one of the first states to use statewide standardized testing as a means of 
assessment, implementing such a testing policy in the early 1 980s. This test was designed to 
ensure that graduates of two-year programs had the basic skills necessary for success and to track 
whether the students were entering the workforce or continuing on to a four-year degree 
program. In 1988, the legislature in State C mandated the assessment of the general education 
curriculum in all two- and four-year institutions. In 1991, the state legislature established a 
series of accountability reporting requirements for State C’s public institutions. 

These requirements were revisited in 1994, when the state legislature articulated a 
stronger link between institutional accountability plans and state educational goals, including 
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access/diversity, quality of undergraduate education, and productivity. The interest in linking 
accountability plans to state educational goals stemmed from the opinion of some state policy 
makers that “existing legislation and institutional responses did not sufficiently embody the kinds 
of characteristics that would lead to improved management at the local level and provide for 
systematic, ongoing assessment.” (Reference: State C’s 1994 Accountability Review Report.) 

State C’s assessment statute has six objectives: (1) to increase accountability to the 
public; (2) to increase fiscal responsibility; (3) to improve teaching; (4) to improve student 
learning; (5) to promote planning on campuses; and (6) to improve academic program efficiency. 
However, the state academic officer reports only one outcome: an increase in 
accountability to the public. No reason was given by the academic officer for this striking 
divergence between objectives and outcomes. Data from the institutions may shed some light on 
the causes of this divergence. 

Institutional Perceptions of Policy 

Table 15 presents the institutional perceptions of the state policy for respondents in State 
C. Five of the 26 public institutions who responded to the ISSA survey do not believe that there 
is a state plan for assessment. However, 25 of the 26 institutions responded that meeting state 
reporting requirements is an important purpose for engaging in assessment. Apparently, at least 
four institutions do not believe there is a state plan, yet agree that there are state reporting 
requirements. Of the 21 who believe there is a state plan, 14 institutions reported that the plan 
had been developed jointly and 7 institutions reported that it had been developed by state level 
officials. 
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Table 15 

Institutional Perceptions of State Policy 

Number 

Responding 



Perception N=26 



Meeting state reporting requirements is important reason to conduct assessment 25 

There is a state plan requiring institutions to conduct assessment 21 

• Assessment plan was developed solely by state officials 14 

• Assessment plan was developed jointly between state and campus reps 7 

State plan was reason to initiative assessment 17 

State plan increased involvement with assessment 18 

State plan had negative influence on assessment 1 

State requires evidence that an assessment plan is in place 17 

State requires measurement of state-mandated indicators 25 

State requires use of institutionally-devised indicators 15 

State requires evidence of institutional use of assessment data 10 



Agreement on aspects of the state plan vary. Most (25) believe that state reporting 
requirements include evidence of measurement of state-mandated indicators. Apparently, State 
C has effectively communicated and promoted the use of its mandated indicators. However, only 
17 of the institutions in State C agree that the state requires evidence that an assessment plan is in 
place, 15 respondents believe that the state reporting requirements include use of institutionally- 
devised indicators, and only 10 respondents believe that the state reporting requirements include 
evidence of institutional use. Differing opinions were not split by institutional type, ruling out 
the possibility that the state requires different activities for different types of institutions. There 
is apparently some confusion in State C regarding what the plan actually stipulates. 

Institutional Approaches to Student Assessment 

Respondents to the ISSA survey categorized both the types of assessment data they were 
collecting and the extent to which they collected this data (See Table 16). Institutions in this 
state are collecting data for at least some students on post-college outcomes, but not all 
institutions are collecting data on either affective or cognitive competencies of their current 
students, despite the state mandate for assessment. The cognitive competencies factor includes 
the measurement of general education. The assessment policy in State C mandates the 
assessment of general education. Nevertheless, 35% of our respondents do not measure general 
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education competencies in students, 31% collect this data for some or many students, and only 
35% collect this data for all students. 



27 



Table 16 

Approaches to Student Assessment in State C 

Mean for Public Institutions Mean for all 

Approach in State C Public Institutions 



Cognitive Assessment 1.43 1.62 

Affective Assessment 1.69 1.74 

Post-college Assessment 2.24* 2.27 

Number of Instruments Used 8 9 



Student-Centered Methods 1.20 1.30 

Externally-Oriented Methods 2.01** 2.06 

Number of Studies Conducted 3 2 

*The scale for these three factors is 1 = not collected, 2 = collected for some students, 3 = collected for many 
students, and 4 = collected for all students. 

**The scale for these two factors is l=not used, 2=used in some units, 3=used in most units, 4=used in all units. 



In addition to traditional instruments, the ISSA survey asked institutions if they used 
student-centered methods for assessment, including portfolios, performance in capstone courses, 
and observations. The ISSA instrument also asked about the use of externally-oriented 
assessment methods, such as interviews with employers and alumni. Institutions are more likely 
to be using external than student-centered methods for assessment. 



Institutional Support for Student Assessment 

Table 17 presents information on institutional support for student assessment in State C. 
All of the respondents from State C reported that both improving faculty instructional 
performance and improving undergraduate student performance were important purposes for 
conducting assessment. Apparently, institutions share the goals of the SHEEO to improve both 
teaching and learning. 
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Table 17 

Support for Student Assessment in State C 


Percentage of 
Institutions in State C 


Percentage of 
all Public Institutions 


Support For Assessment 


N=26 


N=885 


Refer to assessment in mission 


28% 


23% 


Engage in assessment to improve faculty performance 


100% 


93% 


Engage in assessment to improve student achievement 


100% 


98% 


Formally adopted policy requiring assessment in all units 


40% 


51% 


Faculty governance committee addresses student assessment 


25% 


50% 


Faculty governance committees are supportive of assessment 


58% 


65% 


Students are supportive of assessment 


42% 


33% 


Provide a line item operating budget allocation to support 
assessment 


58% 


50% 


Conference funds are available for faculty 


80% 


87% 


Workshops are offered for faculty 


67% 


79% 


Faculty required to receive assessment training 


42% 


60% 


Faculty assistance is offered (i.e. paid leaves, stipends, course 
reductions) 


62% 


53% 



Slightly more than half (58%) of the respondents in State C reported that their faculty 
governance committees were supportive of student assessment. Slightly less than half (42%) of 
the respondents reported that their students were supportive of student assessment. While these 
percentages are comparable to those for all public institutions, support for student assessment is 
not pervasive throughout State C. Administrators in State C tend to support student assessment 
through professional development opportunities for faculty in the form of conference funds, 
workshops, and various forms of assistance. However, less than half of the institutions in State C 
require faculty to receive training on student assessment, compared to 60% of all public 
institutions. 

The survey also asked respondents to indicate how student assessment relates to faculty 
evaluation and rewards. Table 18 displays their responses. Even though most institutions in 
State C encourage faculty to assess their students, this percentage is lower than that for all public 
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institutions (84% vs. 93%). This finding is somewhat surprising, given the state’s expectations 
of assessment activity. Furthermore, relatively few institutions in State C tie assessment to 
tenure, promotion, or salary increases. These low percentages are quite similar to those found 
throughout the country for public institutions. One striking difference is that many more 
institutions in State C are considering assessment experience in the hiring process (62% vs. 38% 
for all public institutions). Perhaps this consideration indicates that institutions in State C realize 
that they need to increase their student assessment activities. 



Table 18 



Faculty Evaluation and Rewards in State C 





Percentage of 


Percentage of 




Institutions in 


all Public 


Faculty Evaluation and Rewards Policies 


State C 


Institutions 


Faculty encouraged to assess students 


84% 


93% 


Hiring process considers assessment experience 


62% 


38% 


Scholarship on student assessment considered for tenure 


37% 


43% 


Evaluation for salary increases considers student performance 


37% 


22% 


Faculty receive awards for assessment 


37% 


33% 


Willingness to participate in assessment considered in promotion 


32% 


39% 


Evaluation for promotion considers student performance 


21% 


30% 



Respondents also reported on their student-oriented policies on student assessment. As is 
demonstrated in Table 19, most institutions actively involve students in the assessment process. 
Only slightly more than a third (37%) use incentives to encourage student participation in 
assessment. 



Table 19 

Student-Oriented Policies on Student Assessment 



Student Policies 



Percentage of Percentage of 
Institutions in all Public 

State C Institutions 



Provide students with individual feedback 
Inform students about the purpose for assessment 
Require students to participate in assessment 
Encourage participation with incentives 



87% 


85% 


84% 


88% 


88% 


88% 


37% 


40% 
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Institutional Impacts of Student Assessment 

In addition to exploring institutional approaches to and support for student assessment, 

our survey results allow us to examine the impacts institutions ascribe to their assessment efforts 

Several questions in the survey relate to changing and improving the learning process. Five of 

these variables are listed in Table 20. A majority of institutions in State C are using assessment 

data to create and or modify their academic support services, general education curriculum, 

instructional methods, and other academic programs. These initiatives demonstrate an effort to 

use assessment results to improve the learning process. 



Table 20 

Institutional Actions Using Student Assessment Data 

Action Taken/Data on Action Taken/Data on 

Assessment Influential Assessment Influential 

Institutional Actions in State C for all Public Institutions 

Modify academic support services 73% 63% 

Revise gen ed curriculum 65% 57% 

Modify instructional methods 62% 60% 

Design/reorganize academic programs 54% 62% 

Revise academic mission 42% 41% 



Survey respondents also reported whether they have monitored how student assessment 
impacts various faculty activities. Table 21 displays how respondents in State C answered these 
questions. When institutions monitor the impact of student assessment on these four faculty- 
related variables, they tend to find that their assessment efforts have had positive impacts. 
Several institutional representatives in State C believe that student assessment has led to changes 
in instructional methods used by faculty and the prevalence of discussions of undergraduate 
education. Only a small number of institutions are examining whether assessment is impacting 
faculty in terms of their interest in teaching and their satisfaction in general. 




32 



31 



Table 21 



Student Assessment Impacts on Faculty Activities in State C 



Number of 





Number of 


Institutions 




Institutions 


Finding Positive 


Impact 


Monitoring 


Impact 


Changes in Instructional Methods Used 


13 


11 


Discussions of Undergraduate Education 


10 


8 


Faculty Interest in Teaching 


7 


5 


Faculty Satisfaction 


4 


2 


Survey respondents also reported how engaging 


in student assessment had impacted 


student performance. Table 22 summarizes these responses. Of those institutions that are 
monitoring, not all are finding that their assessment initiatives have positive impacts on student 
achievement. Less than half of the institutions in State C are attempting to link their assessment 


activities to student grades. 






Table 22 

Student Assessment Impacts on Student Performance 


Number of 


Number of 
Institutions 




Institutions 


Finding Positive 


Impact 


Monitoring 


Impact 


Student retention or graduation rates 


16 


9 


Student achievement on external exams 


11 


8 


Student grade performance 


11 


7 


Student satisfaction 


5 


3 



State C Summary 

Whereas improving teaching and learning are both reported as assessment policy 
objectives by the state officer in State C, neither is reported as a policy outcome. Yet all 26 
institutions responding to the ISSA reported that improving faculty instructional performance 
and improving undergraduate student performance were important reasons for conducting 
student assessment. Clearly, there is agreement between the state and the institutions on the 
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importance of improving teaching and learning. Beyond this agreement, however, there are a 
number of discrepancies. 

While the state has a strong assessment mandate in place, only 25 % of responding 
institutions in State C reported a faculty committee that addresses assessment on their campuses. 
The percentage of State C’s institutions that encourage faculty to assess students, while at 84%, 
still lags behind the national average of 93%. This is all despite the fact that State C has drawn 
clear links between student assessment and state educational goals. State C’s assessment policy 
also mandates the assessment of general education programs at all public institutions. But 35% 
of responding institutions do not collect data on students’ general education competencies. 

In terms of faculty rewards and incentives for engaging in assessment work, less than 
40% of institutions in State C consider assessment when making salary and/or promotion 
decisions. Yet 62% of State C’s institutions indicate on the survey that they look at assessment 
experience when hiring. This is an interesting disconnect: assessment is a factor in hiring but not 
in promoting or compensation. Even more interesting is the fact that State C’s average of 62% in 
this area is well above the national average of 38%. Perhaps as institutions bring more and more 
people with assessment experience into the state, there will be an increased value assigned to 
such activity. 

Conclusions 

There are at least three main themes that emerge from the preceding analysis that have a 
direct bearing on the future of assessment at the state and institutional levels: (1) the extent to 
which there is agreement between the states and the institutions on the importance of improving 
teaching and leaming;(2) the extent to which there is confusion, at the institutional level, about 
state assessment requirements; and (3) the extent to which there is explicit evidence of 
institutional commitment to assessment. 

Agreement 

Based on our survey results, there does seem to be one point on which states and 
institutions concur: the importance of improving teaching and learning as a purpose of 
assessment activity. Despite the strong state mandates in these three states, institutions seem to 
believe that improving teaching and learning is an important reason for engaging in assessment. 
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Most institutions also responded that meeting state guidelines and reporting information to the 
state were important reasons to engage in assessment. 

Confusion 

Despite the agreement that meeting state reporting requirements is important, there was 
considerable confusion and uncertainty about the nature of state assessment requirements. This 
uncertainty manifested itself in a variety of ways. Most fundamentally, there was some doubt 
about whether state assessment requirements existed at all. In State A, two of twelve institutions 
reported that there was no state assessment plan, while in State C, five of twenty-six states 
reported no state assessment plan. (In State B, the confusion— on this point, at least— seems to 
have been eliminated; all ten institutions reported a state assessment plan.) 

Although these percentages are not large (17% in State A and 19% in State C), they 
reflect that institutional understanding, or awareness, of state assessment plans is not universal, 
and at least some of the institutions are not “getting the message” the state is sending. And even 
if institutions agree that a state assessment plan exists, they do not necessarily agree on what the 
plan requires; in all three states, institutional reports diverged rather dramatically on the plan’s 
specifics. In State B, institutions disagreed about whether the state plan called for state- 
mandated or institutionally-driven indicators, and in States A and C, institutions varied in their 
perceptions concerning whether the state plan required evidence of assessment activities. 

Confusion about the manner in which the assessment plan was developed is also 
widespread. In all three states, there was disagreement among institutions about whether the 
assessment plan was developed exclusively at the state level, or if the state and its institutions 
had worked together in crafting the assessment plan. There is some uncertainty regarding the 
effects of a state-required assessment plan. These findings support Ewell & Boyer’s (1988) 
findings that unclear communication was a significant problem in states with mandated 
assessment plans. 

Commitment 

It is, of course, essential that there is institutional commitment to a state-mandated 
assessment plan. The extent to which institutions in these three case study states have committed 
themselves to assessment varied somewhat. In States A and C, one-quarter of institutions 
reported the presence of faculty committees on their campuses that addressed assessment, while 
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the figure stood at one-half for State B. In all 3 states, at least 25% of the respondents said their 
faculty governance committees were not supportive of assessment and at least 50% said their 
students were not supportive of assessment. The extent to which assessment can impact the 
teaching and learning process may be limited by the lack of widespread support for assessment 
on these campuses. Finally, while professional development opportunities exist in all 3 states, in 
each state, fewer than 40% of the respondents said they evaluate and/or reward faculty for 
assessment. This reluctance on behalf of administrators to tie evaluations and rewards to student 
assessment activities and results may indicate a lack of commitment to assessment. 

Implications 

The preceding analysis has implications for the formulation and implementation of 
assessment policy in public higher education. The first and most important of these policy 
implications is the need for improved communication and understanding between states and 
institutions regarding assessment. In some cases, it may be that a state legislature develops an 
assessment plan that is misinformed and poorly constructed. If so, communication from 
institutions to the legislature has broken down; they have not provided the legislature with the 
necessary information to craft an efficient and effective assessment plan. Legislatures need the 
input, and the feedback, from institutions to know what will and will not work in an assessment 
plan, and to secure the involvement of the institutions in the implementation of the plan. After 
all, regardless of the quality of the assessment plan, institutions must “buy in” to it or the plan 
will not work. 

In other cases, institutions may not be clearly articulating what they are accomplishing to 
the legislature. The state officer in one of our three states observed that institutions could not 
describe or explain the connection between assessment and any “concrete” signs of 
improvement, and thus the legislators mistakenly “assumed nothing was happening.” In an era 
of increasing fiscal and political accountability for every public tax dollar spent, state legislators 
are anxious to ensure that funds allocated to higher education are spent proficiently. 

Legislatures, in turn, must telegraph their intentions clearly to institutions. What data 
does the legislature want the institutions to provide in their assessment reports? Are legislators 
more concerned with fiscal efficiency than the improvement of undergraduate education? Is the 
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top priority given to students’ scores on standardized examinations? Will funding be linked to 
institutional performance, and if so, in what way? Legislators and institutional leaders will not 
always agree on these and other related issues, but the key is to make certain each side knows 
what the other side thinks. There may even be, as there were in each of the case study states 
analyzed in this paper, institutions that view the state-mandated assessment plan as adverse to 
their own assessment activities. If that is the case, these concerns should be addressed before 
more tension and ill will are generated. 

As mentioned earlier, an assessment plan is only as good as the institutional commitment 
to the plan. Without the support of the faculty, an assessment plan is virtually doomed to failure. 
In the case study states examined here, no more than one-half of any state’s institutions had 
faculty committees that addressed assessment, and at least 25% of faculty committees and 50% 
of students are not supportive of assessment. These figures assume even greater significance 
when it is remembered that each of these states does, in fact, have high-profile, state-mandated 
assessment plans. The lack of institutional assessment activity, or discernible commitment to 
assessment, in all three states comes as something of a surprise. 

Finally, it is interesting to observe that in all three states, institutions report that the 
assessment they are conducting has a greater impact on improving teaching than on improving 
student learning. This difference in impacts may simply be a function of institutions tending to 
measure changes in teaching more than they attempt to gauge changes in student performance. 
Related to this point is the matter of grades. Grades are traditionally the most common and 
obvious means of measuring student learning; if such learning were improving, higher grades 
should reflect this. But very few institutions are linking grades to assessment, which is 
somewhat surprising. Perhaps greater linkages between grades and assessment would result in 
more tangible evidence that assessment does, in fact, have an impact on learning. Further 
research in this area would be particularly useful. 
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